Stereo in the Presence of Specular Re ection

Stereo in the Presence of Specular Re ection Dinkar N. Bhat Shree. K. Nayar Department of Computer Science Department of Computer Science Columbia University Columbia University New York, NY 10027 New York, NY 10027 [email protected] [email protected] Abstract The problem of accurate depth estimation using stereo in the presence of specular re ection is addressed. Specular reection, a fundamental and ubiquitous re ection mechanism, is viewpoint dependent and can cause large intensity di erences at corresponding points, resulting in signi cant depth errors. We analyze the physics of specular re ection and the geometry of stereopsis which lead to a relationship between stereo vergence, surface roughness, and the likelihood of a correct match. Given a lower bound on surface roughness, an optimal binocular stereo con guration can be determined which maximizes precision in depth estimation despite specular re ection. However, surface roughness is di cult to estimate in unstructured environments. Therefore, trinocular con gurations, independent of surface roughness, are determined such that at each scene point visible to all sensors, at least one stereo pair can produce correct depth. We have developed a simple algorithm to reconstruct depth from the multiple stereo pairs.